Methods and apparatus for query formulation

ABSTRACT

To the standard inverted index database, a new “To” operator is added. The “To” operator treats the standard single-level linear collection of records as being organized into localized clusters. Techniques for hierarchical clusters are presented. During indexing, hierarchical clusters are serialized according to a uniform visitation procedure. Serialization produces bit maps, one for each hierarchical level, that preserve the hierarchical level of each record and its location in the serialization sequence. Also presented are techniques, when searching for an Object-of-Interest, for greatly improving the process by which Exclude Terms are identified. Exclude Terms are particularly useful when the lexical units, representing an Object-of-Interest, are ambiguous. When in the mode of searching for Exclude Terms, the Object-of-Interest can match anywhere in a snippet, rather than just in the focus sentence. Using the “To” operator, the focus sentences thus found are converted into role values, from which are identified candidate Exclude Terms.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is related to the following U.S. patent application(s),which are herein incorporated by reference in their entirety:

“Method and Apparatus For Frame-Based Search,” filed 2008/07/21 (y/m/d),having inventors Wei Li, Michael Jacob Osofsky and Lokesh PooranmalBajaj and App. No. 12177122 (“the '122 Application”);

“Method and Apparatus For Frame-Based Analysis of Search Results,” filed2008/07/21 (y/m/d), having inventors Wei Li, Michael Jacob Osofsky andLokesh Pooranmal Bajaj and App. No. 12177127 (“the '127 Application”);

“Method and Apparatus For Determining Search Result Demographics,” filed2010/04/22 (y/m/d), having inventors Michael Jacob Osofsky, Jens ErikTellefsen and Wei Li and App. No. 12765848 (“the '848 Application”);

“Method and Apparatus For HealthCare Search,” filed 2010/05/30 (y/m/d),having inventors Jens Erik Tellefsen, Michael Jacob Osofsky, and Wei Liand App. No. 12790837 (“the '837 Application”); and

“Method and Apparatus For Automated Generation of Entity Profiles UsingFrames,” filed 2010/07/20 (y/m/d), having inventors Wei Li, MichaelJacob Osofsky and Lokesh Pooranmal Bajaj and App. No. 12839819 (“the'819 Application”).

Collectively, the above-listed related applications can be referred toherein as “the Related Applications.”

FIELD OF THE INVENTION

The present invention relates generally to the clustered storage ofinformation, and more particularly to efficiently representinghierarchical information within a framework of records searchable by aninverted index. The present invention also relates generally to theformulation of queries and more particularly to the selection ofexclusion terms.

BACKGROUND OF THE INVENTION

Inverted Index Databases (or IIDBs) are well known. An example IIDB isthe well-known Open Source software “Lucene,” that uses an invertedindex to perform rapid searches of a collection of records (Lucene isprovided by “The Apache Software Foundation,” a not-for-profit Delawarecorporation, with a registered office in Wilmington, Del., U.S.A.).IIDBs like Lucene are sufficiently efficient and scalable such that theycan be used for searching a large-scale corpus, a function provided byweb-accessed search engines.

A limitation of IIDBs like Lucene is that the only inherent structuralrelationship supported, between records, is the single-level linearcollection. It should be noted that the basic item of indexed data,supported by generic Lucene (i.e., Lucene that lacks the presentinvention), is called a “document.” However, herein, for purposes ofgenerality, we shall refer to the basic item of indexed data as a“record.” Each record of an IIDB is identified by a unique ID number(where Lucene currently has capability to store up to 2³¹ records, sincethe unique ID for each record is a 32 bit signed integer).

It would therefore be desirable to augment IIDBs to permit efficientrepresentation of structural relationships, between records of an IIDB,that are more complex than just a single-level linear collection.

An important use of IIDBs is the searching of a “Corpus of Interest” (orC_of_I) for mentions of an “Object of Interest” (or O_of_I). Aparticular type of O_of_I is a brand of consumer products (also referredto herein as a “Consumer Brand” or “C_Brand”). C_Brands can be thesubject of large-scale database searches, particularly of Internetcontent, by Brand Managers (persons responsible for the continuedsuccess of a C_Brand). In particular, a Brand Manager is ofteninterested, for example, in the sentiment of consumers toward his or herC_Brand.

The names of many C_Brands, however, can be ambiguous.

Ambiguity, in a lexical unit, means that the same lexical unit can havetwo or more distinctly different meanings. Some example C_Brands, withambiguous names, include the following:

-   -   “Tide”:        -   C_Brand meaning: a laundry detergent        -   Example alternate meanings:            -   the tide of the ocean            -   a football team, called “Alabama Crimson Tide”    -   UPS:        -   C_Brand meaning: a package-delivery service        -   a Example alternate meaning: a direction of motion away from            the earth    -   Visa:        -   a C_Brand meaning: a credit card company        -   a Example alternate meaning: a official document allowing            entry into a foreign nation

It would therefore be highly desirable to provide techniques for theformulation of queries that are more precise at the identification of anO_of_I (such as a C_Brand), while still achieving a high level ofrecall.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, that are incorporated in and constitute apart of this specification, illustrate several embodiments of theinvention and, together with the description, serve to explain theprinciples of the invention:

FIGS. 1A-1B graphically depict the hierarchical structure of first andsecond clusters.

FIGS. 2A-2B show a fragment of a single-level linear collection, ascould be maintained by an IIDB.

FIG. 3 shows an example representation, using four bit maps, of thebasic structural information of the 15 records of FIGS. 2A and 2B.

FIGS. 4A-4B depict, in a general way, two main modes by which bit mapscan be used.

FIG. 5 shows an example pseudo-coded “To” procedure.

FIGS. 6A-6B show an example pseudo-coded “Single_To” procedure.

FIGS. 7A-7F are a diagrammatic representation of the reconstructedportions of a cluster that can be produced, by successive iterations ofa “while” loop, when mapping towards higher levels of the cluster.

FIGS. 8A-8H are a diagrammatic representation of the reconstructedportions of a cluster that can be produced, by successive iterations ofa “while” loop, when mapping towards lower levels of the cluster.

FIG. 9 depicts an example production-level computer system design inwhich the techniques described herein can be applied.

FIG. 10A depicts a query-entry screen 1000, for the investigation ofC_Brands.

FIG. 10B depicts a screen 1010, with the frame-based search resultsproduced by investigation of the C_Brand “tide” for positive emotions.

FIG. 10C shows an example screen 1020, of the Exclude Term Assistant,when a Brand Manager is seeking to find Exclude Terms to improve asearch for the Consumer Brand “tide.”

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Reference will now be made in detail to various embodiments of theinvention, examples of which are illustrated in the accompanyingdrawings. Wherever possible, the same reference numbers will be usedthroughout the drawings to refer to the same or like parts.

Please refer to the Section 4 (“Related Applications”) and Section 5(“Glossary of Selected Terms”), included as the last two sections of theDetailed Description, for the definition of selected terms used below.

Table of Contents to Detailed Description 1 Clustered Storage  1.1 FirstEmbodiment  1.2 Pseudo-code   1.2.1 Mapping Up The Hierarchy  1.2.2 Mapping Down The Hierarchy 2 Query Formulation  2.1 ConsumerSentiment Search  2.2 Exclude Term Assistant   2.2.1 Search for LexicalUnit of Interest   2.2.2 Focus Sentences to Role Values  2.2.3 Down-sampling   2.2.4 Frequency and Cluster Analysis  2.2.5 User Selection of Exclude Terms 3 Computing Environment4 Related Applications 5 Glossary of Selected Terms

1 CLUSTERED STORAGE 1.1 First Embodiment

A first embodiment of the present invention permits efficientrepresentation of structural relationships, between records of an IIDB,that are more complex than just a single-level linear collection. Inparticular, the present invention is directed to data storage situationswhere, at a local level, it is useful to view records as being organizedinto clusters. A “cluster” can encompass any kind of data organizationthat relies on relatively localized connections between its records. Thestructural organization for clusters focused upon herein is thehierarchical (or “tree”) structure, but it should be understood that thepresent invention can be applied to clusters with other organizationalarrangements.

The two basic types of operations, provided by an IIDB that lacks thepresent invention, are as follows. For simplicity of explanation, theoperations are described as producing, or operating upon, ordered listsof record ID numbers. However, any suitable data structure, thatprovides functionally similar results, can be used.

-   -   1. Query Operations. Querying refers to the collection of        operations where one or more keywords are matched, against an        inverted index, in order to produce an ordered list of record ID        numbers. Each member of the ordered list satisfies the query. In        addition to permitting queries for individual keywords, a query        language typically provides operators by which keywords can be        required to have certain positions relative to each other        (called herein “Positional” operators). For example, putting a        sequence of two or more keywords within a pair of quotes often        represents a kind of Positional operation. The pair of quotes        indicate that the keywords within must all be found next to each        other, and in the same order, for the expression to be        satisfied. Another type of Positional operator is often referred        to as the “Within” operator. A “Within” operator can take three        arguments: a first argument that is a keyword (or a sequence of        keywords, as indicated by a pair of quotes), a second argument        that is a keyword (or a sequence of keywords, as indicated by a        pair of quotes), and a third argument that specifies a maximum        allowable distance, measured in words, between the first and        second arguments.    -   2. Boolean combination, between ordered lists of record ID        numbers, to produce a resulting ordered list of record ID        numbers. Since Boolean operators produce and take as input the        same type of data representation (e.g., an ordered list), they        can be composed into arbitrarily complex expressions. Example        Boolean operators include the following: AND, OR, and NOT.

To the two above-listed basic operations, the present invention adds anintra-cluster level-conversion capability, referred to herein as a “To”operator. Before explaining the “To” operator, it is useful to explainhow localized clusters of data can be encoded within a single-levellinear collection of records.

FIGS. 2A-2B, for example, show a fragment of a single-level linearcollection, as could be maintained by an IIDB. This fragment consists ofRecords 0-14, where the ellipses, following Record 14, indicate thatmany more records could be part of the collection.

Records 0-14 have been divided into the following two clusters ofrecords:

-   -   1. A first cluster of records, each identified with an ID number        from the range 0-8, is shown in FIG. 2A.    -   2. A second cluster of records, numbered 9-14, is shown in FIG.        2B.

FIGS. 1A and 1B, respectively, graphically depict the hierarchicalstructure of the first and second clusters. The hierarchy shown has thefollowing four levels (proceeding from top level to bottom level):

-   -   1. Snippet    -   2. Sentence    -   3. Instance    -   4. Role

While any collection of symbols can be used to denote the levels of ahierarchy, it is often useful to assign a unique integer to each level.For purposes of discussion herein, we shall follow the convention ofassigning a zero to the highest level and increment, by one, the valuerepresenting each successive level. Therefore, the four levels discussedjust-above can be represented as follows:

Snippet: 0 Sentence: 1 Instance: 2 Role: 3

As can be seen from the assignment of record ID numbers, in FIGS. 1A and1B, each cluster is converted into a linear collection of records byvisiting its records in a depth-first manner. The process, of convertinga cluster of records into a linear collection of records, is also calledherein “serialization.” While the clusters of FIGS. 1A and 1B wereserialized by visiting their records in a depth-first manner, anysuitable visitation procedure can be used.

Even without the “To” operator, there are still a variety of operationsthat can be performed upon an IIDB of serialized clusters. For example,consider the four-level hierarchical clusters discussed just-above. Eachlevel of the hierarchy can be given its own “class” of record, and eachsuch class can be accessible by its own selection of indexed fields. Anexample class structure follows:

-   -   1. SnippetObj: Each record of type “SnippetObj” represents a        snippet. Permits searching, by snippet content, at an indexed        field called “Snippet.”        -   1.1. SentenceObj: Each record of type “SentenceObj”            represents a sentence of its parent snippet.            -   Permits searching, by sentence content, at an indexed                field called “Sentence.”            -   Permits searching, for the focus sentence, at an indexed                field called “Focus.” The content of the “Focus” field                can be either “True” or “False.”        -   1.1.1. InstanceObj: Each record of type “InstanceObj”            represents an instance, for each frame found in its parent            sentence. Permits searching, by frame type, at an indexed            field called “FrameType.”            -   1.1.1.1. RoleObj: Each record of type “RoleObj”                represents a role of its parent instance. Permits                searching, by role value content, at indexed field                “Value.”                (In the above class structure, it can be seen that the                class names include an “Obj” suffix. This suffix is used                to indicate a relationship to Object-Oriented                Programming. The suffix is not intended to indicate, as                that term is discussed in below Section 2, “Query                Formulation,” that they are “Objects of Interest” for                which a user desires to formulate a query.)

Using the above-defined class structure, following are four example IIDBsearch queries (where each query is followed by explanatory commentary):

-   -   SnippetObj.Snippet=‘Tide’        -   Finds all snippets that have the word “Tide”    -   SentenceObj.Focus=‘True’        -   Finds all the focus sentences of all snippets    -   InstanceObj.FrameType=‘Like’        -   Finds all instances of the “Like” Frame    -   RoleObj.Value=‘crimson tide’        -   Finds all role values that contain the phrase “crimson tide”

Further, Boolean operators can be used to combine search queries. Forexample, the following query finds all focus sentences that have theword “Tide”:

-   -   AND(SentenceObj.Focus=‘True’, SentenceObj.Sentence=‘Tide’)

To the above, the “To” operator adds the ability for a query result, atone level of a cluster hierarchy, to be translated into its equivalentform at another level the cluster hierarchy.

The “To” operator receives (either explicitly or implicitly) at leasttwo parameters:

-   -   1. First parameter: an ordered list of record ID numbers. Each        ID number is interpreted as representing a record at a same        particular level in a cluster. If the clusters are hierarchical,        the level of a record “n” is typically the least number of edges        to traverse, from the root, to reach “n.” There is a common        level, for each ID number of the first parameter, and it is        called a “start level.” For example, with respect to FIGS. 2A        and 2B, assume the first parameter, input to a “To” operator, is        the following list of record ID numbers: (4, 11). In that case,        each ID number represents an “Instance” level record (because        Record 4 represents Instance 130 and Record 11 represents        Instance 131).    -   2. Second parameter: a target level. Can be expressed in an        absolute or relative way.

For each record “r” of the first parameter, the “To” operator does thefollowing:

-   -   1. the cluster “c,” to which “r” belongs is identified;    -   2. the record or records of “c” that correspond to “r,” at the        target level, are identified.

If the clusters are hierarchical, there are two main possibilities:

-   -   1. target level>start level: In this case, there can be >1        records, at the target level, each of which is descendent of        “r.” Being a descendent of “r” can also be referred to as being        contained within the scope of “r.”    -   2. target level<start level: In this case there can only be one        record, at the target level, that is an ancestor of “r.” An        ancestor of “r” can also be referred to as the record, at the        target level, that contains “r” within its scope.

To continue with the same example of ID numbers (4, 11), discussedjust-above, an example target level, for these instances, could be the“role” level. Conversion of the ID numbers (4, 11), from the instance torole level, produces the following list of ID numbers: (5, 6, 12, 13).In the form of an IIDB query expression, the example can be expressed asfollows:

-   -   To (+1, InstanceObj.FrameType=‘Like’)        The above “To” expression can be analyzed as follows:    -   The second parameter is the ordered list of record ID numbers.        For the particular expression shown above, it is the set of all        instances of the “Like” Frame.    -   The first parameter specifies the target level. For the above        expression, the first parameter is assumed to be relative and        going one step lower in the hierarchy (i.e., going from parent        to child) is assumed to increase the level number by one.        The two parameters work together as follows: for each record        indicated by the second parameter, convert its child records        (which are the equivalent record or records at the next level        lower for each cluster's hierarchy).

The above example “To” expression can be related, as follows, to theabove example of going from (4, 11) “To” (5, 6, 12, 13). We will assumethat Record 4 (or instance 130), of FIGS. 1A and 2A, is an instance ofthe “Like” Frame. We will also assume that Record 11 (or instance 131),of FIGS. 1B and 2B, is an instance of the “Like” Frame. Thus, going tothe next lower level, for Record 4, means finding Records 5 and 6, whilegoing to the next lower level, for Record 11, means finding Records 12and 13.

Conversion of ID numbers, from their starting level in a hierarchy tothe equivalent ID numbers at a target level, requires that at least thefollowing basic structural information is preserved in theserialization:

-   -   1. The level, in the hierarchy, of each record. In FIGS. 2A and        2B, this is known from the labels kept with each record. For        example, for the example Record ID numbers (4, 11), each such        Record is labeled as an “instance” (abbreviated “INSTAN”).    -   2. The location of each record, in the serialization sequence.

The two above-listed types of information, along with knowing thevisitation procedure (e.g., depth-first) by which a serializationsequence was produced, is sufficient to accomplish the “To” operation.To make the “To” operation as fast as possible, so that it can beapplied (for example) to search results produced from the querying ofvery large databases, both types of basic structural information can bepreserved with bit maps.

FIG. 3 shows an example representation, using four bit maps, of thebasic structural information of the 15 records of FIGS. 2A and 2B. Ascan be seen, each level of the hierarchy (snippet, sentence, instance,role) is represented with its own bit map. Each bit in a bit map (oreach “column” across the set of four bit maps) corresponds, by virtue ofits position, to a record ID number. Regarding the above-discussed list(4, 11), it can be seen that each record ID number is represented (inFIG. 3) by a “1” bit, in the “instance” bit map, at the columns forrecords 4 and 11.

If the clusters are hierarchical, FIGS. 4A-4B depict, in a general way,the two main modes by which bit maps are used. For each of theseFigures, the following conventions are used:

-   -   Each bit, of a bit map, is represented by a circle.    -   Each horizontal dotted line represents a bit map.    -   The hierarchical level, represented by each bit map, increases        from bottom to top.    -   Record ID number increases from left to right.    -   The bits, representing the start level records, are processed        from right to left. In the diagrams, this is labeled the “Inter        start record order.”

A discussion, of each of FIGS. 4A-4B, follows:

-   -   1. target level<start level: Illustrated by FIG. 4A. In this        case each bit, of “start level” bit map, is pursued up the        hierarchy of its cluster, until its single ancestor, at the        target level, is found. In particular, FIG. 4A shows two bits        413 and 412, at the start level. For each of these bits, its        ancestor bit, respectively bits 411 and 410, is found. An        ancestor bit is found by a right to left traversal of the bit        maps (labeled in the FIG. 4A as the “Intra start record order”).    -   2. target level>start level: Illustrated by FIG. 4B. In this        case each bit, of “start level” bit map, is pursued down the        hierarchy of its cluster, until its descendent bit or bits, at        the target level, are found. In particular, FIG. 4A shows two        bits 415 and 414, at the start level. For each of these bits,        its descendent bits (418 and 419 for 415; 417 and 416 for 414)        are found. A descendent bit is found by a left to right        traversal of the bit maps (labeled in FIG. 4B as the “Intra        start record order”).

1.2 Pseudo-Code

Example pseudo-code, for interpreting bit maps of similar structure tothat shown in FIGS. 3 and 4A-4B, is depicted in FIGS. 5 and 6A-6B.

Line 1 of FIG. 5 shows that the example pseudo-coded “To” proceduretakes the following two parameters:

-   -   1. rel_target_level: A target level in cluster hierarchy, for        each record of the input list of records, to be mapped to. For        the version of the “To” operator shown herein in the        pseudo-code, the target level is input as a relative value: the        target level is determined by adding rel_target_level (which can        be either positive or negative) to the “start level.” As        discussed above, the start level is the common level, in each        cluster's hierarchy, for each record of the input list of        records. (A will be seen below, the non-relative start level is        represented, in the pseudo-code, by the variable “start level”        of FIGS. 6A-6B.)    -   2. recs_(—)2b_mapped: The input list of records to be mapped.

The “To” procedure returns (see line 18 of FIG. 5) a list of recordsthat have been mapped to the target level. A line-by-line discussion ofFIG. 5 (and its supporting procedure “Single_To”) follows.

FIG. 5 handles the two main cases of the “To” procedure as follows:

-   -   1. target level<start level: Addressed by FIG. 5, lines 5-12.    -   2. target level>start level: Addressed by FIG. 5, lines 13-16.

For either case, the “To” procedure calls the “Single_To” procedure(either at line 10 or 14 of FIG. 5). Line 1 of FIG. 6A shows that theexample pseudo-coded “Single_To” procedure takes the following threeparameters:

-   -   1. rel_target_level: Same meaning as for “To” procedure        discussed above.    -   2. start_rec: Set to a single record (selected by “To” from the        input list of records) to be mapped. Also referred to as the        “start record.”    -   3. addi_results: This parameter is useful as an efficiency        measure, when Single_To is asked to map the start record to its        higher level ancestor record. Such ancestor record is also        referred to herein as the “target record.” The target record can        also be the ancestor record to other records of the “To”        procedure's input list. As will be discussed further below,        Single_To finds the target record by reconstructing at least        part of the original hierarchy (between the start record and the        target record) that had been serialized into the IIDB. Once        Single_To has found the target record it can also check the        reconstructed hierarchy to see whether such hierarchy contains        any additional records, of the same level as the start record,        for which target record is also the ancestor record. Such        additional records are returned through the addi_results        parameter.

The “Single_To” procedure returns (see line 23 of FIG. 6A and line 19 ofFIG. 6B) a list of records that have been mapped to the target level.“Single_To” is similar to “To” in the sense that it is also structuredinto two main sections, depending upon which, target level or startlevel, is greater:

-   -   1. target level<start level: Addressed by FIG. 6A, lines 6-24.        In this case, for the input start record, the “Single_To”        operator is seeking the one record, higher in a cluster's        hierarchy, that contains it (also called its ancestor record).    -   2. target level>start level: Addressed by FIG. 6B, lines 1-20.        In this case, we start with an ancestor record, and the        “Single_To” operator seeks those one or more records, at some        lower level in a cluster's hierarchy, that are contained by such        ancestor record.        A line-by-line discussion of “Single_To” follows.

1.2.1 Mapping Up the Hierarchy

For purposes of example, let us first assume that Single_To has beeninvoked as follows on the IIDB of FIGS. 2A-2B:

-   -   1. relative target level=−3    -   2. start record=Record 6    -   3. additional results=null

Single_To starts by initializing the start and target levels (FIG. 6A,lines 2-3). The function “level” can operate, for example, by consultingthe bit maps at the record ID number indicated by the record passed toit as a parameter. For a start record of Record 6, the “level” function(of line 2) can examine column 6, of the bit maps of FIG. 3, anddetermine that its level is “3” (also known as the “role” level). Thetarget level is then set to “0.”

Next, Single_To prepares for its reconstruction, of at least part of acluster's serialized records, by setting the current record to the startrecord (FIG. 6A, line 4). This means, for our example, that current_recpoints to Record 6.

If the target level is less than the start level, cluster reconstructionis performed by the “while” loop of lines 7-21. For the example, since0<3, this “while” loop is executed.

The “while” loop will perform a first execution, of the loop body, ifits test (line 7) is satisfied. In our example, since 3≠0, a firstiteration is begun.

For purposes of the pseudo-code, serialization is assumed to be done ina depth-first “left to right” order. Thus, reconstructing a cluster'shierarchy in an upwards fashion, when starting from the start record,involves a step-by-step accessing of records in a “leftwards” direction.In the example, this ordering can be seen when viewing FIGS. 2A-2B and3. To begin the upwards reconstruction, the record to the left (the“left record” or “left_rec”) of the current record is found (line 8).For the example, left_rec is set to Record 5. The left record issubjected to three tests:

-   -   1. level of left record=level of current record (line 9): If        this test is satisfied, we know that the left record is a        sibling of the current record. For the example, we can see        (particularly clearly in FIG. 1A) that Record 5 and 6 are        siblings.    -   2. level of left record<level of current record (line 11): If        this test is satisfied, we know that the left record is the        parent of the current record. Also, since we do not know what        iteration of the “while” is being executed, we need to check        whether the left record is the parent of any other records,        already visited, that are at the same level as the current        record.    -   3. level of left record>level of current record (line 16): If        this test is satisfied, we know that the left record is        starting, with respect to the current record, a new sub-tree of        the cluster's hierarchy. Also, because of the depth-first        serialization, we know that the current record will be a sibling        of any other records found at the same level of the current        record.

Now that the current record has been processed, the left record is madethe new “current record” (line 20) and the second iteration of the“while” is started. A diagrammatic representation, of the reconstructionproduced by the first “while” iteration, when processing the example, isshown in FIG. 7A.

Successive iterations of the “while” loop (of FIG. 6A, lines 7-21) areperformed, until the level of the current record is the same as thetarget level. For the example, diagrammatic representations, ofsuccessive iterations, are shown in FIGS. 7B-7F.

FIGS. 7A-7F represents each record, with respect to FIG. 1A, insimplified form: each record is labeled only with its record ID number.Where two or more records are marked as being siblings, for purposes oftheir later connection through a parent, this is indicated by theabbreviation “SIB.”

Briefly, each of the following iterations can be described as follows:

-   -   Iteration 2:        -   current record=Record 5        -   left record=Record 4        -   level of left record (level 2)<level of current record            (level 3), so Records 5 and 6 are made children of Record 4            (see FIG. 7B)    -   Iteration 3:        -   current record=Record 4        -   left record=Record 3        -   level of left record (level 1)<level of current record            (level 2), so Record 4 is made child of Record 3 (see FIG.            7C)    -   Iteration 4:        -   current record=Record 3        -   left record=Record 2        -   level of left record (level 1)=level of current record            (level 1), so Records 2 and 3 are marked as siblings (see            FIG. 7D)    -   Iteration 5:        -   current record=Record 2        -   left record=Record 1        -   level of left record (level 1)=level of current record            (level 1), so Records 1 and 2 are marked as siblings (see            FIG. 7E)    -   Iteration 6:        -   current record=Record 1        -   left record=Record 0        -   level of left record (level 0)<level of current record            (level 1), so Records 1-3 are made children of Record 0 (see            FIG. 7F)

When attempting to start iteration 7, since the current record is Record0, the condition of the “while” is not satisfied (because level (Record0)=target_level). Therefore, the “while” loop ends with thereconstruction being shown in FIG. 7F. As can be seen from FIG. 7F,“addi_results” is set to Record 5 (by line 22 of FIG. 6A) and Record 0is the returned value (by line 23 of FIG. 6A).

Thus, “Single_To” has mapped Record 6 to Record 0 and Record 5 isavailable as an additional result. This additional result could beuseful, for example, if the “To” procedure was called with an input list(or “recs_(—)2b_mapped”) that included both Records 5 and 6. As can beseen from FIG. 5, the “To” procedure steps through the input list (the“for” loop at line 4) in a direction reverse to serialization. Thus,Single_To would be called with Record 6 as its parameter before Record5. Following the call to Single_To with Record 6, the “for” loop wouldset rec_current to Record 5. In this case, the “if” of line 6 (of FIG.5) is satisfied, resulting in Record 5 being immediately added (by line7) to the result to be returned, without calling Single_To. Then(because of line 8 and its “go directly” command) the “for” loop canproceed directly to the next record (of recs_(—)2b_mapped) to beprocessed.

1.2.2 Mapping Down The Hierarchy

In order to explain the portion of the Single_To procedure, illustratedby FIG. 6B, it is assumed that it has been invoked as follows on theIIDB of FIGS. 2A-2B:

-   -   1. relative target level=+3    -   2. start record=Record 0    -   3. additional results=null        This is essentially the opposite of the example discussed above        for FIG. 6A.

As was the case with the previous example of mapping up the hierarchy,Single_To starts by initializing the start and target levels (FIG. 6A,lines 2-3). For Record 0, the “level” function can examine column 0, ofthe bit maps of FIG. 3, and determine that its level is “0” (also knownas the “snippet” level). The target level is then set to “3.”

Next, the Single_To procedure prepares for reconstruction, of at leastpart of a cluster's serialized records, by setting the current record tothe start record (FIG. 6A, line 4). This means, in the example, thatcurrent_rec points to Record 0.

If the target level is greater than the start level (see “if” of FIG.6B, line 1), cluster reconstruction is performed by the “while” loop ofFIG. 6B, lines 3-18. As was discussed above, for purposes of the “To”and “Single_To” pseudo-code, serialization is assumed to have been donein a depth-first “left to right” order. Thus, reconstructing a cluster'shierarchy in a downwards fashion, starting from a start record, involvesstep-by-step accessing of records in a “rightwards” direction. To beginthe downwards reconstruction, the record to the right (the “rightrecord” or “right_rec”) of the current record is found (FIG. 6B, line2). The “while” loop (line 3) ends when the right record (the nextrecord to be analyzed) is at a same or higher level, in the hierarchy,than the start level.

For the example, the “if” of FIG. 6B, line 1 is satisfied (since 3>0).The right record is set to Record 1, and each iteration of the loopexecutes as follows. (In the following listing of iterations, it shouldbe noted that FIGS. 8A-8F follow a similar formatting to FIGS. 7A-7F:each record, with respect to FIG. 1A, is shown in simplified form whereit is labeled only with its record ID number.)

-   -   Iteration 1:        -   current_rec=Record 0        -   right_rec=Record 1        -   Since level of Record 1>level of Record 0 (line 4), the            right record is made a child of the current record. This            reconstruction is shown in FIG. 8A.    -   Iteration 2:        -   current record=Record 1        -   right_rec=Record 2        -   Since level of Record 2=level of Record 1, Record 2 is made            a sibling of Record 1. This reconstruction is shown in FIG.            8B.    -   Iteration 3:        -   current record=Record 2        -   right_rec=Record 3        -   Since level of Record 3=level of Record 2, Record 3 is made            to be a sibling of Record 2. This reconstruction is shown in            FIG. 8C.    -   Iteration 4:        -   current record=Record 3        -   right_rec=Record 4        -   Since level of Record 4>level of Record 3, the right record            is made a child of the current record. This reconstruction            is shown in FIG. 8D.    -   Iteration 5:        -   current record=Record 4        -   right_rec=Record 5        -   Since level of Record 5>level of Record 4, the right record            is made a child of the current record. This reconstruction            is shown in FIG. 8E.    -   Iteration 6:        -   current record=Record 5        -   right_rec=Record 6        -   Since level of Record 6=level of Record 5, Record 6 is made            to be a sibling of Record 5. This reconstruction is shown in            FIG. 8F.    -   Iteration 7:        -   current record=Record 6        -   right_rec=Record 7        -   While the level of the right records jumps to a smaller            value, from 3 to 1, it is still greater than the start            level, so the “while” loop continues.        -   Since level of Record 7<level of Record 6, line 6 is            satisfied. Further, since there is not equality, between the            levels of Records 6 and 7, the “else” of line 9 is            performed.        -   Since Record 0 is the most recently created record that, in            the reconstructed tree, is higher than Record 7, Record 7 is            made a child of Record 0 (see FIG. 8G).    -   Iteration 8:        -   current record=Record 7        -   right_rec=Record 8        -   Since level of Record 8=level of Record 7, Record 8 is made            to be a sibling of Record 7. This reconstruction is shown in            FIG. 8H.    -   Preparation for iteration 9:        -   current record=Record 8        -   right_rec=Record 9        -   Since level of Record 9 (see FIGS. 1B and 3) is equal to the            start level, the “while” loop ends, leaving FIG. 8H as the            final reconstructed tree.

In terms of returning a value for “Single_To,” line 19 returns thoserecords (Records 5 and 6) of the reconstructed tree that are at thetarget level (of level 3).

2 QUERY FORMULATION

2.1 Consumer Sentiment Search

Distinguishing a usage of a lexical unit that is intended to refer to an“Object of Interest” (O_of_I), from a usage of a lexical unit that isintended to refer to something other than the O_of_I, can be greatlyassisted by the inclusion of “Exclude Terms” in a search query.

In general, an Exclude Term can be defined as follows. It is a term thatcan be included as part of a query where, if the term is found in arecord of an IIDB, that record is excluded from inclusion in the searchresult.

The present invention, for the formulation of Exclude Terms forinclusion in a database query, can be applied to any “Corpus ofInterest” (C_of_I) for which mentions, of an O_of_I, are to beidentified. A particular type of search, to provide an example where thepresent invention can be utilized, is presented in this Section 2.

The particular type of O_of_I is a brand of consumer products (alsoreferred to herein as a “Consumer Brand” or “C_Brand”). C_Brands can bethe subject of large-scale database searches, particularly of Internetcontent, by Brand Managers (persons responsible for the continuedsuccess of a C_Brand). In particular, a Brand Manager is ofteninterested, for example, in the sentiment of consumers toward his or herC_Brand.

A C_of_I can be collected and searched for mentions of the O_of_I. Inthe case of a C_Brand, an example C_of_I can be a database thatrepresents the collection, in a large scale and comprehensive way, ofpostings (such as “tweets” on Twitter) to Social Media (SM) web sites orservices. We can refer to such Social Media database as “SM_db.”

As has been described in the above-referenced Related Applications(please see Cross Reference to Related Applications), a frame-basedsearch tool can be provided, by which instances of an O_of_I can besought, in a C_of_I, in connection with a particular type of concept orconcepts. More particularly, a Brand Manager can be provided with aframe-based search tool by which instances of a C_Brand can be sought,in a SM_db, in connection with a particular type of concept. Forpurposes of example herein, the “concept” presented is that of aconsumer expressing the fact that he or she “likes” a C_Brand.

An example set of roles, for a “Like” frame, are as follows (each rolename is in capitals, with a brief explanation following):

-   -   AGENT: The entity that expresses the “liking.”    -   OBJECT: The object (for example a C_Brand) towards which the        “liking” is expressed.    -   EMOTION: A particular positive emotion, if any, expressed by the        Agent towards the Object.    -   BEHAVIOR: A particular positive behavior, if any, expressed by        the Agent towards the Object.    -   ASPECT: A particular positive quality or property, if any,        expressed by the Agent towards the Object.

The above “Like” frame is typically applied to the analysis of anindividual sentence (referred to herein as the “focus sentence”).Following is an example focus sentence, to which appropriate NaturalLanguage Processing (NLP) can be applied to produce an instance of the“Like” frame. The following example sentence discusses a fictitiousbrand of soda called “Barnstorm”:

-   -   “My children love Barnstorm Soda and buy it all the time because        of its taste.”

Given a suitable NLP analysis, by application of suitable frameextraction rules, the following instance of the “Like” frame can beproduced:

-   -   AGENT: “My children”    -   OBJECT: “Barnstorm Soda”    -   EMOTION: “love”    -   BEHAVIOR: “buy”    -   ASPECT: “taste”

In addition to the focus sentence, each post to Social Media can besummarized as a three sentence “snippet,” with the focus sentenceforming the middle sentence. A single type of record, let us call it“SentenceObj,” can include both the focus sentence and the snippet asfields. These fields can be called, respectively, “FocusSentence” and“Snippet,” with each field being indexed and therefore available forqueries. Thus, when searching for all SentenceObjs, that satisfy aparticular query, there are at least two indexes that can be used. As anexample, if the FocusSentence index is to be searched for alloccurrences of the word “Tide” and the Snippet index is to be searchedfor all occurrences of the word “government,” then an IIDB syntax, forexpressing these queries, can be (respectively) as follows:

-   -   SentenceObj.FocusSentence=‘Tide’    -   SentenceObj.Snippet=‘Government’

In a similar manner to that discussed above (Section 1, “ClusteredStorage”), each SentenceObj record, of the SM_db, can part of a separate“cluster” of an IIDB. Including to the SentenceObj, the cluster can behierarchically organized to contain the following three record types:

-   -   2. SentenceObj: Primary purpose is to represent focus sentence        of snippet.        -   Permits searching, by focus sentence content, at indexed            field “FocusSentence.”        -   Permits searching, by snippet that includes the focus            sentence, at indexed field “Snippet.”    -   2.1.InstanceObj: Represents an instance, for each frame found in        focus sentence. Permits searching, by frame type, at indexed        field “FrameType.”        -   2.1.1. RoleObj: Represents the roles, for each instance.            Permits searching, by role value content, at indexed field            “Value.”            This hierarchy of three record types can be referred to            herein as the “Consumer Sentiment Hierarchy” or “CSH.” To            improve readability, of query expressions included below            that use the CSH, “comments,” in the style of the C            Programming Language, may be included.

As has been discussed in the Related Applications, a large-scaledatabase (such as the SM_db), and its indexes, is typically createdbefore a user query is input. The IIDB of the present application,however, can differ from those discussed in the Related Applicationsbecause of the cluster storage invention of above Section 1 (“ClusteredStorage”).

A user can formulate a query by identifying lexical unit or unitsrepresentative of the O_of_I (e.g., a C_Brand). For purposes of example,it is assumed that the O_of_I is identified by only one lexical unit.Further, for purposes of example, we will address the C_Brand called“Tide” (a brand of laundry detergent) and assume it is to be identified,by a Brand Manager, by the single lexical unit “Tide.”

All focus sentences, of the SM_db, can be searched for usage of thelexical unit “Tide.” Based upon the CSH, the following query can returnall resulting focus sentences:

-   -   SentenceObj.FocusSentence=‘Tide’

If a user has already identified an Exclude Term, to be used inconjunction with the search query, it can also be utilized. All focussentences, where its snippet contains at least one Exclude Term, can beexcluded from the search results. For example, based upon the CSH, thefollowing query can return the list of all focus sentences where itssnippet contains the Exclude Term “Government”:

-   -   SentenceObj.Snippet=‘Government’

Using the “NOT” operator, according to the following expression, theabove ordered list (of focus sentences where its snippet contains theExclude Term) can be converted to the list of focus sentences where itssnippet does not contain the Exclude Term:

-   -   NOT(SentenceObj.Snippet=‘Government’)

Finally, the Exclude Term can be applied to the above-listed search for“Tide” with the AND operator:

-   -   AND(        -   NOT(SentenceObj.Snippet=‘Government’),        -   SentenceObj.FocusSentence=‘Tide’)    -   )

An example user interface, for entering this type of search is shown inFIG. 10A. FIG. 10A depicts a query-entry screen 1000, for theinvestigation of C_Brands, that contains the following items:

-   -   Search term entry box 1001: one or more lexical units, each of        which if found in a focus sentence is regarding as indicating a        possible occurrence of a C_Brand, are entered here. In the        diagram, only the single lexical unit “Tide” is shown. A search,        based on the lexical units of box 1001, can be initiated by a        user selecting the “Search” button 1003 with mouse pointer 1050.    -   Exclude term entry box 1002: one or more lexical units, each of        which if found in a focus sentence's snippet is used to exclude        that focus sentence, are entered here. In the diagram, only the        single lexical unit “Government” is shown. The following section        (Section 2.2) describes a tool, called “Exclusion Term        Assistant” (or ETA), that can be used to help identify Exclude        Terms. Use of this ETA can be initiated by a user selecting the        “ETA” button 1004 with mouse pointer 1050.

For each focus sentence found, the “To” operator (discussed above inSection 1, “Database Storage”) can be applied twice:

-   -   1. To go from the set of focus sentences “To” the instance of        the “Like” frame (if any) found to have occurred in such        sentence.    -   2. For each “Like” instance found, to go from the instance “To”        a particular role or roles of such instance.

For purposes of example, we will assume that the role of interest, forthe “Tide” brand, is “Emotion.” It is assumed that the Brand Managerwishes to know all the positive emotions consumer associate with “Tide.”

Going from the set of focus sentences “To” the set of all instanceswithin such focus sentences can be accomplished with the followingexpression:

-   -   To (        -   +1,        -   AND(            -   NOT(SentenceObj.Snippet=‘Government’),            -   SentenceObj.FocusSentence=‘Tide’)        -   ) /* end AND */    -   ) /* end First “To” */

The set of all instances of the “Like” frame can be found from thefollowing:

-   -   InstanceObj.FrameType=‘Like’

The set of instances, within the selected focus sentences, can belimited to those of the “Like” frame by combination of the above-twoexpressions with the AND operator:

-   -   AND (        -   To (            -   +1,            -   AND(                -   NOT(SentenceObj.Snippet=                -   ‘Government’),                -   SentenceObj.FocusSentence=‘Tide’            -   )        -   ), /* end First “To” */        -   InstanceObj.FrameType=‘Like’    -   ) /* end outer AND */

The role values, of the “Like” frames, can be found by use (as describedabove) of a second “To” in the following expression:

-   -   To (        -   +1,        -   AND (            -   To (                -   +1,                -   AND(                -    NOT(SentenceObj.Snippet=                -    ‘Government’),                -    SentenceObj.FocusSentence=                -    ‘Tide’                -   )            -   ), /* end First “To”*/            -   InstanceObj.FrameType=‘Like’        -   ) /* end outer AND */    -   ) /* end Second “To”*/        For clarity of explanation, the above expression can be        represented, symbolically, by the identifier        “ROLE_VALUES_SEARCH_RESULT.”

The different role values found can be subjected to grouping analysis.In grouping analysis, similar roles values can be put into a singlegroup and the group given a generic name (or “g_name”). For theparticular example, the following are several role values that can beplaced in a common group:

-   -   love    -   really love    -   really always love

The common lexical unit, among a group of role values, can beidentified. For the example above, the common lexical unit is “love.”Thus, all three role values can be presented to the user (e.g., theBrand Manager) as a single emotion of interest “love.”

Grouping can continue recursively, with subgroups being identifiedwithin a group. For the example above, it can be seen that “really love”can be identified as a subgroup of “love.”

In addition to placing role values into groups, and presenting genericrole value names (or g_name's) to the user, the order of presentation ofsuch g_name's can be determined by the frequency with which each suchg_name appears in the search result. Thus, for example, if the g_name“like” represents 953 focus sentences (where each of the 953 focussentences contains at least one occurrence of the word “like”), and theg_name “love” represents only 262 focus sentences, then the g_name“like” is presented before the g_name “love.”

An example user interface, for presenting such g_names is shown in FIG.10B. FIG. 10B depicts a screen 1010, for the investigation of C_Brands,that contains the following sub-panes:

-   -   Sub-pane 1011: Contains two columns, 1012 and 1013. Column 1012        lists the example g_names discussed above, for the emotions        “Like,” “Love,” and “Thrill.” Column 1013 lists, for each        corresponding g_name, the number of focus sentences in which it        appears. Further emotions can be listed by a user selecting,        with mouse pointer 1050, the link “view more emotions.”    -   Sub-pane 1012: Lists five of the focus sentences (1014-1018)        found by the search (such as the search query of FIG. 10A).        Viewing the focus sentences can help a user to better evaluate        the precision of his or her search, by showing the search term        within various contexts. For example, focus sentences 1014 and        1017 do appear to be statements of interest to a Brand Manager,        while the other focus sentences appear to discuss other meanings        of the word “tide.” To make it easier to assess the relevance of        each focus sentence shown, the search term can be shown with any        appropriate graphical emphasis (e.g., any or all of underlining,        boldfacing, color highlighting). For focus sentences 1014-1018,        “Tide” is emphasized by boldfacing and underlining. Further        focus sentences can be listed by a user selecting, with mouse        pointer 1050, the link “view more focus sentences.”

When a user selects a g_name, it can be useful to see its usage incontext. In other words, in can be useful to see at least a sampling ofthe focus sentences in which the g_name appears. For FIG. 10B, forexample, this could correspond to the user selecting g_name “Like” andthen seeing, in pane 1012, a listing of focus sentences that express theemotion “like.” From the perspective of IIDB queries, this operation canbe accomplished as follows:

-   -   1. The search result (represented symbolically by        “ROLE_VALUES_SEARCH_RESULT”) can be searched for all role values        that use the user-selected g_name. The result of this search can        be referred to as the ordered list “g_name_list.” Based on the        CSH, and an example role value being sought of “Like,” the IIDB        query can be:        -   AND(            -   ROLE_VALUES_SEARCH_RESULT,            -   RoleObj.Value=‘Like’        -   )    -   2. The “To” operator is then applied to g_name_list, in order to        translate its list of role-level role values to the higher level        of focus sentences that contain such role values. Since the        focus sentences are two levels higher than the role values, in        the CSH, the IIDB query can be:        -   To(            -   −2,            -   AND(                -   ROLE_VALUES_SEARCH_RESULT,                -   RoleObj.Value=‘Like’            -   )        -   )

When displaying such focus sentences to the user, it can be useful tohighlight (or otherwise emphasize), within each such focus sentence, theoccurrence of the g_name that caused the sentence to be displayed.

2.2 Exclude Term Assistant

While the above-described search process of Section 2.1 can often bevery useful, it can have certain limitations. An example is the searchof a SM_db for mentions of a C_Brand where the lexical unit or units,that represent the C_Brand, are ambiguous.

For the Section 2.1 example, of a Brand Manager of the C_Brand “Tide”searching a SM_db, an example query with an exclude term was alreadydiscussed. It is the following expression that excludes the word“government”:

-   -   AND(        -   NOT(SentenceObj.Snippet=‘Government’),        -   SentenceObj.FocusSentence=‘Tide’    -   )

The present invention provides techniques for greatly improving theprocess by which Exclude Terms are identified. A step-by-steppresentation of a process, that illustrates these techniques, follows.

2.2.1 Search for Lexical Unit of Interest

The Exclude Term identification process begins with a search for thelexical unit or units, that can refer to the O_of_I in the C_of_I. Forclarity of explanation, we shall refer to one lexical unit, aspotentially referring to the O_of_I. We can refer to that one lexical asthe lexical unit of interest (or LU_of_I). (It is clear to anyone ofordinary skill in the art, that the following procedure can be expandedto accommodate more than one LU_of_I.)

The LU_of_I is assumed to be ambiguous, and therefore have at least twomeanings:

-   -   1. A meaning that refers to the O_of_I.    -   2. A meaning that refers to something distinctly different from        the O_of_I.

We will continue with the above-described example of a Brand Manager,seeking to research a C_Brand in a SM_db. In particular, we will use theexample of the LU_of_I being “Tide,” and the single Exclude Term of“government” having been identified, as is shown in FIG. 10A.

By selecting the Exclude Term Assistant (ETA) button 1004, of FIG. 10A,the appropriate IIDB (such as the SM_db), can be searched for usage ofthe LU_of_I (e.g., “Tide”). However, unlike Section 2.1, rather thanlimiting a match, to the LU_of_I, to being in a focus sentence, a focussentence will be included in a search result so long as the LU_of_Imatches anywhere in the focus sentence's snippet. A broader search isdone in the present section because we are specifically seeking ExcludeTerms. Therefore, we are specifically interested in finding contexts,other than the desired context of interest, where the same LU_of_I maybe utilized. In a similar manner to that discussed in Section 2.1, thefollowing query can return the list of all snippets that contain theLU_of_I:

-   -   SentenceObj.Snippet=‘Tide’

Any Exclude Terms, already identified, can be stored in stored in a listreferred to herein as the “Exclude Term List” (or “ET_Iist”). Whenperforming the search of the snippets, even if a snippet has theLU_of_I, if the snippet also includes a member of the ET_Iist, then thesnippet is not included in the search result. For the case of “n”Exclude Terms, and the LU_of_I being “Tide,” the following expressioncan be used to produce a list of snippets that is reduced by each memberof the ET_Iist:

-   -   AND(        -   SentenceObj.FocusSentence=‘Tide’,        -   NOT(SentenceObj.Snippet=ET_list₁),        -   NOT(SentenceObj.Snippet=ET_list₂),        -   . . .        -   NOT(SentenceObj.Snippet=ET_list_(n))    -   )

For clarity of explanation, the list of focus sentences resulting fromthis step can be referred to as “ETA_FOCUS_SENTENCES.”

2.2.2 Focus Sentences to Role Values

For each focus sentence, from the set of focus sentences retrieved(i.e., ETA_FOCUS_SENTENCES), the “To” operator can be used twice (in amanner similar to that discussed above in Section 2.1, “ConsumerSentiment Search”):

-   -   1. To go from the set of focus sentences “To” the instances of        the frames (such as the “Like” frame) found to have occurred in        such sentence. For clarity of explanation, the list of instances        resulting from this use of the “To” operator can be referred to        as “ETA_INSTANCES.”    -   2. For each instance found (such as an instance of the “Like”        frame), to go from the instance “To” the role values of a        particular role or roles of such instance. For clarity of        explanation, the list of resulting from this use of the “To”        operator can be referred to as “ETA_ROLES.”

However, unlike Section 2.1, the two just-above listed uses of the “To”operator can differ, respectively, as follows:

-   -   1. First use of the “To” operator can be used to go to many more        types of frame instances than just the particular frame the user        may have an interest in (such as a Brand Manager user just being        interested in the “Like” frame). Once again (as was discussed        above in Section 2.2.1 “Search for Lexical Unit of Interest”)        this broader mapping is done because we are specifically seeking        Exclude Terms.    -   2. Second use of “To” operator is typically used to focus on        roles where the role value denotes a kind of “object.” For        example, for the above-discussed “Like” frame, this is the        Object role. This focus, upon roles that denote an object, is        done because the purpose of the Exclude Terms is to remove, from        a search result, objects that are not the O_of_I.

2.2.3 Down-Sampling

Given the size of the IIDB's that can be processed, there may be toomany resulting role values from the previous step (i.e., the role valuesindicated by ETA_ROLES), for processing by the next step of frequencyand cluster analysis. For example, the step of Section 2.2.2 can produce20-30 million role values.

A sampling can be performed, of only a portion of the result ofETA_ROLES, to produce a computationally tractable number of role values.Any of the known statistical techniques, for approximating the range ofvalues of a larger population from a smaller sample of that population,can be used. For example, assume that ETA_ROLES represents an orderedlist of 10⁷ role values and that only 10⁵ values can be processed, bythe next step (Section 2.2.4), in a sufficiently small time period. Thismeans that for each 10² role values, of ETA_ROLES, only 1 is included inthe set of role values passed-along by this step for further processing.

Regardless of whether down-sampling is preformed, for purposes of thenext step (Section 2.2.4), it is assumed that ETA_ROLES indicates theordered list of roles for processing.

2.2.4 Frequency and Cluster Analysis

Given the ordered list of role values produced by either of Section2.2.2 or Section 2.2.3 (either of which is indicated by ETA_ROLES), thefollowing four main steps can be performed. Collectively, the followingfour steps can be referred to herein as “Basic Frequency and ClusterAnalysis” or “Basic FCA”:

-   -   1. Each role value of ETA_ROLES can have each of its constituent        lexical units put into a generic form (by such operations as        “stemming”) and “stop words” can be eliminated.    -   2. Frequencies of occurrence of unique sets of one or more        lexical units, across role values of ETA_ROLES, is determined.        -   a. To illustrate this step 2 (as well as the rest of Basic            FCA) an example, referred to herein as “Example Frequency            and Cluster Analysis” or “Example FCA,” is presented. For            Example FCA, we will consider only the following small            subset of role values for ETA_ROLES: (“tide”, “crimson            tide”, “crimson tide”, “high tide”, “crimson tide”, “high            tide”)        -   b. The unique sets of one or more lexical units are:            (“tide”), (“crimson”), (“high”), (“tide”, “crimson”),            (“tide”, “high”).        -   c. The unique sets, each with its frequency of occurrence            among the example role values, are:            -   i. Unique Set: (“tide”) Frequency: 6            -   ii. Unique Set: (“crimson”) Frequency: 3            -   iii. Unique Set: (“high”) Frequency: 2            -   iv. Unique Set: (“tide”, “crimson”) Frequency: 3            -   v. Unique Set: (“tide”, “high”) Frequency: 2    -   3. The sets of lexical units can be subjected to grouping        analysis, where similar sets of lexical units can all be put        into a single group. The group can be given a group name based        upon the overlap, of lexical units, between sets. The frequency        of occurrence of a group name can be the frequency of occurrence        of its corresponding set.        -   a. Continuing with Example FCA, the following groups can be            formed:            -   i. Group: “tide” Frequency: 6                -   1. Unique Set: (“tide”) Frequency: 6                -   2. Unique Set: (“tide”, “crimson”) Frequency: 3                -   3. Unique Set: (“tide”, “high”) Frequency: 2            -   ii. Group: “crimson” Frequency: 3                -   1. Unique Set: (“crimson”) Frequency: 3                -   2. Unique Set: (“tide”, “crimson”) Frequency: 3            -   iii. Group: “high” Frequency: 2                -   1. Unique Set: (“high”) Frequency: 2                -   2. Unique Set: (“tide”, “high”) Frequency: 2    -   4. The group names and/or lexical unit sets can be listed in        order of decreasing frequency. These group names and lexical        unit sets represent candidate Exclude Terms and can be presented        to a user for selection.        -   a. Continuing with Example FCA, the following listing can be            formed:            -   i. Group: “tide” Frequency: 6                -   1. Unique Set: (“tide”, “crimson”) Frequency: 3                -   2. Unique Set: (“tide”, “high”) Frequency: 2            -   ii. Group: “crimson” Frequency: 3                -   1. Unique Set: (“tide”, “crimson”) Frequency: 3            -   iii. Group: “high” Frequency: 2                -   1. Unique Set: (“tide”, “high”) Frequency: 2        -   b. As can be seen, the lexical unit set that just contains            the word “tide” is both the most frequent (having 6            occurrences) and overlaps with the following two other            lexical unit sets: (“tide”, “crimson”) and (“tide”, “high”).            Within these two other lexical unit sets, (“tide”,            “crimson”) occurs more frequently than (“tide”, “high”) and            therefore (“tide”, “crimson”) is listed first.

The above-described Basic FCA, along with the Example FCA, can berelated, as follows, to FIG. 10C. FIG. 100 shows an example screen 1020,of the ETA, when a Brand Manager is seeking to find Exclude Terms toimprove a search for the Consumer Brand “tide.”

FIG. 10C is divided into the following three panes:

-   -   1. Candidate Exclude Terms Pane 1021: Depicts the results of        subjecting ETA_ROLES to the above-described Basic FCA. Pane 1021        has three main columns:        -   a. Check box column 1024: Provides a check box interface, by            which a user can select a candidate Exclude Term as an            actual Exclude Term.        -   b. Candidate Exclude Terms column 1025: Lists, in order of            decreasing frequency, the candidate Exclude Terms. As shown            in column 1025, the list of candidates can be grouped.            Specifically, “tide” is shown to be a group name for six            narrower (and multi-word) candidate Exclude Terms. These six            multi-word candidates are indicated as being grouped under            “tide” by their indenting. However, any suitable graphical            technique, for the indication of grouping, can be used.        -   c. Frequency column 1026: Presents, for a corresponding            candidate

Exclude Term, an indication of its frequency of occurrence among thefocus sentences found (i.e., ETA_FOCUS_SENTENCES). Column 1026 depictsthe percentage of focus sentences, with respect to ETA_FOCUS_SENTENCES,in which the corresponding candidate Exclude Term occurs. While thepercentage is presented numerically, any suitable graphical technique,for indicating a relative amount, can be used. Further, while a relativemeasure is shown, any other suitable measure of frequency can be used.For example, an absolute count, of the number of actual occurrences of acorresponding candidate Exclude Term, can be shown.

-   -   2. Summary Pane 1022: Presents summary information, regarding        the status of the user's query. Pane 1022 shows (from top to        bottom) the following three example items of summary information        (but any suitable forms of summary information could be        included):        -   a. Total number of focus sentences found, before using            Exclude Terms as found by the ETA.        -   b. Number of focus sentences found, when using Exclude Terms            as found by the ETA. Also indicates, by a relative measure            (such as percentage), the extent to which the number of            focus sentences found has been reduced.        -   c. Number of candidate Exclude Terms specifically selected            by user (such specific selection is discussed below in            Section 2.2.5).    -   3. Focus Sentences Pane 1023: Depicts a selection of focus        sentences from ETA_FOCUS_SENTENCES.

Column 1025 has certain similarities to the Example FCA. As is shown forthe Example FCA, for the fourth step of Basic FCA, column 1025 alsoshows “tide” as a group name, with “crimson tide” and “high tide” beingsub-members of that group. Also like the Example FCA, column 1025 shows“crimson tide” listed before “high tide” because of the greaterfrequency of “crimson tide.” Specifically, frequency column 1026 shows“crimson tide” and “high tide” as having, respectively, relativefrequency indicators of 8% and 6%.

Although not specifically shown in column 1025, as illustrated in FIG.10C, it can be assumed that the “crimson” group name, if opened, wouldshow “crimson tide” as a sub-member (and this sub-member is found in theExample FCA at the fourth step of Basic FCA).

The term “high” as a group name, as found in the Example FCA at thefourth step of Basic FCA, is not illustrated in FIG. 10C.

Frequency and cluster analysis has been described as being performed onthe values of the roles, that correspond to the focus sentences found bysearch of Section 2.2.1. However, frequency and cluster analysis couldbe performed directly upon the focus sentences found by search ofSection 2.2.1. In this case, candidate Exclude Terms can be found bydetermining the various n-grams, to a suitable level of “n,” on thefocus sentences.

2.2.5 User Selection of Exclude Terms

As has already been introduced as a topic above, in Section 2.2.4, theuser can select one or more items, from the list of candidate ExcludeTerms. Such selection can be based upon any combination of the followingfactors (including other factors not specified herein)

-   -   1. Position of a candidate in the list, such position indicative        of its frequency and/or importance, in relation to other        candidate Exclude Terms.    -   2. Statistical information, listed with one or more of the        candidate Exclude Terms. Such statistical information is        typically informative of the frequency of the candidate with        which it is listed. The statistical information can be listed in        any suitable form or format, including numerical or graphical.    -   3. Compiled or summary information, regarding the list of        remaining focus sentences, after application of the selected        Exclude Terms. The list of remaining focus sentences is        typically measured relative to the list (ETA_FOCUS_SENTENCES)        produced by the search of Section 2.2.1. For example, as a        result of selection of one or more candidates, it can be        displayed that only 80% of the focus sentences remain.    -   4. Precision information, regarding the nature of the focus        sentences associated with a candidate Exclude Term. For example,        when a candidate Exclude Term is selected, a window can display        a selection of the focus sentences in which it appears. A user        can scroll through such window to better evaluate the extent to        which such focus sentences are off-topic.

With specific reference to FIG. 10C, we see that the user hasspecifically selected the following four candidate Exclude Terms:“crimson tide,” “high tide,” “rise tide,” and “alabama crimson tide.”This selection is indicated by the check box in column 1024, for each ofthese candidate Exclude Terms, containing the symbol for a “check mark”(✓). These four selections also lead to a showing of the following:

-   -   “4” Exclude Terms, in pane 1022, with “Num Exclude Terms.”    -   Only 80%, of the focus sentences ETA_FOCUS_SENTENCES, being left        after using the Exclude Terms. As can be seen from column 1026,        the percentages, of the four selected Exclude Terms, add up to        20%.    -   The check box for group name “tide,” in column 1025, containing        a black square. A black square in a check box, as opposed to a        “check mark” symbol, indicates that only some (but not all) of        the candidate Exclude Terms, under a group name, have been        selected. In the case of the group name “tide,” only four of its        six narrower candidate Exclude Terms are selected.    -   The check boxes for group names “alabama” and “crimson,” in        column 1025, containing a black square. We can assume that each        of “alabama” and “crimson” contain narrower candidate Exclude        Terms, but that such narrower terms are not shown in FIG. 10C.        We can assume that one (but not all) of the narrower terms under        “alabama” is the specifically selected “alabama crimson tide.”        We can make a similar assumption for “crimson.”

Precision information, regarding the nature of the focus sentencesassociated with a candidate Exclude Term, can be available in screen1020 of FIG. 10C. Pane 1021 shows the user having placed his or hermouse pointer 1050 over the candidate Exclude Term “tide free.” Inresponse, a dashed outline around “tide free” is shown, indicating thatthis term is being considered but has not yet been selected. In responseto being considered, focus sentences pane 1023 displays a selection ofthe focus sentences in which the candidate Exclude Term (in thisexample, “Tide Free”) appears. To assist the user in observing thecontext for “Tide Free,” the candidate Exclude Term can be graphicallyemphasized using any suitable technique (pane 1023 emphasizes eachappearance of “Tide Free” with underlining). A user can scroll throughsuch window, to obtain a better understanding of the extent to whichsuch focus sentences are off-topic. In the case of “Tide Free,” it canbe seen that since such focus sentences are probably not off-topic,since they relate to a detergent. Therefore, it is probably the casethat the user does not want to use “Tide Free” as an Exclude Term.

The user can loop back, to the beginning of the Exclude Term selectionprocess, by returning to the search of Section 2.2.1. Upon suchloop-back, the search will differ by inclusion the Exclude Termsselected. In FIG. 100, this loop-back can be accomplished by the usercan selecting “Update” button 1027.

Alternatively, if the user is satisfied with the quality of the query,he or she can escape the ETA by “closing” screen 1020 (user-interfacefor closing not shown) and returning to a screen such as screen 1000 ofFIG. 10A. Once back to FIG. 10A, but with the necessary Exclude Termspresent, the user can perform (for example) a Consumer Sentiment Search,such as described in Section 2.1.

3 COMPUTING ENVIRONMENT

FIG. 9 depicts an example production-level computer system design inwhich the techniques described herein can be applied.

Cloud 930 represents data, such as online opinion data, available viathe Internet. Computer 910 can execute a web crawling program, such asHeritrix, that finds appropriate web pages and collects them in an inputdatabase 900. An alternative, or additional, route for collecting inputdatabase 900 is to use user-supplied data 931. For example, suchuser-supplied data 931 can include the following: any non-volatile media(e.g., a hard drive, CD-ROM or DVD), record-oriented databases(relational or otherwise), an Intranet or a document repository. Acomputer 911 can be used to process (e.g., reformat) such user-supplieddata 931 for input database 900.

Computer 912 can perform the indexing needed for formation of anappropriate frame-based database (FBDB). FBDB's are discussed in theRelated Applications. The indexing phase scans the input database forsentences that refer to an organizing frame (such as the “Like” frame),produces a snippet around each such sentence and adds the snippet to theappropriate frame-based database. FIG. 9 depicts an example FBDB 901.For the example frame-based search systems described in Section 2, anFBDB based on the “Like” frame could be produced.

Databases 920 and 921 represent, respectively, stable “snapshots” ofdatabases 900 and 901. Databases 920 and 921 can provide stabledatabases that are available for searching, about an O_of_I in a C_of_I,in response to queries entered by a user at computer 933. Such userqueries can travel over the Internet (indicated by cloud 932) to a webinterfacing computer 914 that can also run a firewall program. Computer913 can receive the user query, collect snippet and frame instance datafrom the contents of the appropriate FBDB (e.g., FBDB 921), and transmitthe results back to computer 933 for display to the user. The resultsfrom computer 913 can also be stored in a database 902 that is privateto the individual user. When it is desired to see the snippets, on whicha graphical representation is based, FBDB 921 is available. If it isfurther desired to see the full documents, on which snippets are based,input database 920 is also available to the user.

In accordance with what is ordinarily known by those in the art,computers 910, 911, 912, 913, 914 and 933 contain computing hardware,and programmable memories, of various types.

The information (such as data and/or instructions) stored oncomputer-readable media or programmable memories can be accessed throughthe use of computer-readable code devices embodied therein. Acomputer-readable code device can represent that portion of a devicewherein a defined unit of information (such as a bit) is stored and/orread.

4 RELATED APPLICATIONS

The description presented herein relies on many parts of the RelatedApplications. This section makes reference to particular portions of the'837 Application, which is a member of the group of the RelatedApplications.

In general, sections of the '837 Application can be referred to hereinby the following convention. Where “X” is a section number, the sectioncan be referred to as: Section X, '837. If the title of the section isto be included, where the title is “Title,” it can be referred to as:Section X, '837 (“Title”) or Section X, '837, “Title.”

Section 4, '837 (“FBSE”) describes a Frame-Based Search Engine (orFBSE). This FBSE is a more generic form of the kind of search describedherein in Section 2.1 (“Consumer Sentiment Search”).

Section 4.2, '837 discusses frames as a form of concept representation(Section 4.2.1) and the use of frame extraction rules to produceinstances of frames (Section 4.2.2). A pseudo-code format for frameextraction rules is presented in Section 6.2, '837 (“Frame ExtractionRules”).

Snippets are discussed in Section 6.4, '837.

The “Frame-Based Database” (FBDB), discussed herein in Section 3(“Computing Environment”), is described in Section 4.3.2 (“Pre-QueryProcessing”), '837.

5 GLOSSARY OF SELECTED TERMS

-   ancestor: with respect to a node “y,” used to refer to a node “x,”    at some higher level, that contains node “y.”-   C_Brand: “Consumer Brand”-   C_of_I: “Corpus of Interest”-   CSH: “Consumer Sentiment Hierarchy”-   descendent: with respect to a node “y,” used to refer to a node “z,”    at some lower level, that is contained by node “y.” If “z” is the    descendent of “y,” then “y” is also the ancestor of “z” (see    definition of ancestor).-   ETA: Exclude Term Assistant-   LU_of_I: lexical unit of interest-   list: Used herein to refer to any collection of data items. Defined    herein to refer to any suitable data structure, that provides    functionally similar results.-   O_of_I: “Object of Interest”-   ordered collection: Preserves an ordering among data items and    allows such data items to be referenced as a unit. Defined herein to    refer to any suitable data structure, that provides functionally    similar results.-   ordered list: Preserves an ordering among data items and allows such    data items to be referenced as a unit. Defined herein to refer to    any suitable data structure, that provides functionally similar    results.-   precision: as a search query becomes more “precise,” the likelihood    increases, that any one search result is a desired search result.-   pseudo-code (or pseudocode): as presented herein, is loosely based    upon the syntax and semantics of the C Programming Language.-   recall: as a search query achieves better “recall,” the likelihood    increases that all desired search results are included in the actual    search result (although if precision is low, many undesired search    results may also be included in the actual search result).-   set: Used herein to refer to any collection of data items. Defined    herein to refer to any suitable data structure, that provides    functionally similar results.

While the invention has been described in conjunction with specificembodiments, it is evident that many alternatives, modifications andvariations will be apparent in light of the foregoing description.Accordingly, the invention is intended to embrace all such alternatives,modifications and variations as fall within the spirit and scope of theappended claims and equivalents.

What is claimed is:
 1. A method for query formulation, comprising:searching, performed at least in part with a configuration of computinghardware and programmable memory, a corpus, to produce a firstcollection of records, by testing for appearances of a lexical unit thatcan be indicative of an object-of-interest; identifying, performed atleast in part with a configuration of computing hardware andprogrammable memory, a first collection of role values corresponding tothe first collection of records; performing, performed at least in partwith a configuration of computing hardware and programmable memory,frequency analysis of sets of one or more lexical units, containedwithin the first collection of role values, to produce a first candidatelist of exclude terms; and wherein each role value is part of a frameinstance, each instance produced by application of a frame extractionrule to a record of the corpus.
 2. The method of claim 1, furthercomprising: performing cluster analysis, of the sets of one or morelexical units, to produce a first candidate list of grouped excludeterms.
 3. The method of claim 1, wherein the identifying is performed byuse of an operator that identifies a cluster, for each record of thefirst collection of records, and maps the record to another hierarchicallevel.
 4. The method of claim 1, wherein the searching is performed bytesting for appearances of the lexical unit in snippets.
 5. The methodof claim 1, wherein each record, of the first collection of records,represents a focus sentence.
 6. The method of claim 1, furthercomprising: reducing a number of role values, contained in the firstcollection of role values, by obtaining a sample of the first collectionof role values.
 7. The method of claim 1, further comprising: selectingone or more exclude terms, from the first candidate list of excludeterms.
 8. The method of claim 7, further comprising: searching a corpus,to produce a second collection of records, by testing for appearances ofthe lexical unit and excluding any record that contains a member of thefirst candidate list of exclude terms.
 9. The method of claim 8, whereinthe searching, for both the second first and second collection ofrecords, is performed by testing for appearances of the lexical unit insnippets.
 10. The method of claim 8, wherein the searching, for thefirst collection of records, is performed by testing for appearances ofthe lexical unit in snippets, and the searching, for the secondcollection of records, is performed by testing for appearances of thelexical unit in focus sentences.
 11. The method of claim 1, wherein thefirst collection of roles values is collected from a role that isindicative of an object.
 12. A system for query formulation, comprising:a sub-system configured, as a result of the computing hardware andprogrammable memory, to accomplish searching a corpus, to produce afirst collection of records, by testing for appearances of a lexicalunit that can be indicative of an object-of-interest; a sub-systemconfigured, as a result of the computing hardware and programmablememory, to accomplish identifying a first collection of role valuescorresponding to the first collection of records; a sub-systemconfigured, as a result of the computing hardware and programmablememory, to accomplish performing frequency analysis of sets of one ormore lexical units, contained within the first collection of rolevalues, to produce a first candidate list of exclude terms; and whereineach role value is part of a frame instance, each instance produced byapplication of a frame extraction rule to a record of the corpus. 13.The system of claim 12, further comprising: a sub-system configured, asa result of the computing hardware and programmable memory, toaccomplish performing cluster analysis, of the sets of one or morelexical units, to produce a first candidate list of grouped excludeterms.
 14. The system of claim 12, wherein the identifying is performedby use of an operator that identifies a cluster, for each record of thefirst collection of records, and maps the record to another hierarchicallevel.
 15. The system of claim 12, wherein the searching is performed bytesting for appearances of the lexical unit in snippets.
 16. The systemof claim 12, wherein each record, of the first collection of records,represents a focus sentence.
 17. The system of claim 12, furthercomprising: a sub-system configured, as a result of the computinghardware and programmable memory, to accomplish reducing a number ofrole values, contained in the first collection of role values, byobtaining a sample of the first collection of role values.
 18. Thesystem of claim 12, further comprising: a sub-system configured, as aresult of the computing hardware and programmable memory, to accomplishselecting one or more exclude terms, from the first candidate list ofexclude terms.
 19. The system of claim 18, further comprising: asub-system configured, as a result of the computing hardware andprogrammable memory, to accomplish searching a corpus, to produce asecond collection of records, by testing for appearances of the lexicalunit and excluding any record that contains a member of the firstcandidate list of exclude terms.
 20. The system of claim 19, wherein thesearching, for both the second first and second collection of records,is performed by testing for appearances of the lexical unit in snippets.21. The system of claim 19, wherein the searching, for the firstcollection of records, is performed by testing for appearances of thelexical unit in snippets, and the searching, for the second collectionof records, is performed by testing for appearances of the lexical unitin focus sentences.
 22. The system of claim 12, wherein the firstcollection of roles values is collected from a role that is indicativeof an object.